Information processing apparatus

ABSTRACT

A streaming data acquisition section  302  acquires streaming data distributed from a server and including an application image, and a streaming data reproduction section  322  reproduces the streaming data. A command input acceptance section  342  accepts a command input to an application inputted by a user. A command transmission section  344  transmits the inputted command. The command is provided to an information processing apparatus of a distributor of the streaming data and is reflected on processing of the application being executed in the information processing apparatus.

TECHNICAL FIELD

The present invention relates to a technology for distributing orviewing streaming data of an application of a game or the like.

BACKGROUND ART

As a communication tool for many people, various social networkingservices (SNS) are being utilized. Also a service for sharing a movingpicture prepared or captured by a user has become widespread, and a usercan share various pieces of information with different users on anetwork.

SUMMARY Technical Problem

In recent years, environments in which a plurality of users can play agame together through the Internet have been prepared. Such a networkservice of a game as just described already has a role as acommunication tool, and construction of a mechanism by which informationcan be shared efficiently between friends or with strangers is demanded.Especially, it is expected to make it possible to streaming distribute agame image being played by a user simply and readily such that, throughsharing of a live video of the game with a different user, the fun ofthe game can be spread and communication between users becomes active.It is to be noted that it is preferable for information to be sharedefficiently not only in a game but also in a different type of a networkservice in which a plurality of information processing apparatuses areconnected to each other.

Therefore, it is an object of the present invention to provide atechnology for streaming distributing or viewing image data of anapplication.

Solution to Problem

In order to solve the problem described above, according to a mode ofthe present invention, there is provided an information processingapparatus including a first acquisition unit configured to acquirestreaming data including an application image, an image generation unitconfigured to reproduce the streaming data, an acceptance unitconfigured to accept a command input to an application, and atransmission unit configured to transmit the inputted command.

Also another aspect of the present invention is directed to aninformation processing apparatus. The apparatus includes an executionunit configured to execute an application, an image generation unitconfigured to generate image data to be displayed on an outputtingapparatus, a sharing processing unit configured to transmit the imagedata generated by the image generation unit, and an acquisition unitconfigured to acquire information relating to the image data. The imagegeneration unit generates image data including an application image andan image of the information acquired by the acquisition unit, and thesharing processing unit encodes and transmits the image data to bedisplayed on the outputting apparatus.

It is to be noted that also arbitrary combinations of the constituentelements described above and the representations of the presentinvention obtained by conversion thereof between a method, an apparatus,a system, a recording medium, a computer program and so forth areeffective as modes of the present invention.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram depicting an information processing system accordingto an embodiment of the present invention.

FIG. 2 is a diagram depicting an appearance configuration of aninputting apparatus.

FIG. 3 is a diagram depicting an appearance configuration of theinputting apparatus on the rear face side.

FIG. 4 is a diagram depicting an internal configuration of the inputtingapparatus.

FIG. 5 is a diagram depicting a functional block diagram of aninformation processing apparatus.

FIG. 6 is a diagram depicting an internal configuration of theinformation processing apparatus which operates as a distributionapparatus of streaming data.

FIG. 7 is a diagram depicting an example of a menu screen imagedisplayed on an outputting apparatus.

FIG. 8 is a diagram depicting an example of a game screen imagedisplayed on the outputting apparatus.

FIG. 9 is a diagram depicting an example of an inputting screen imagedisplayed in an overlapping relationship on the game screen image.

FIG. 10 is a diagram depicting an example of an inputting screen imagedisplayed in an overlapping relationship on the game screen image when“Video Streaming” is selected.

FIG. 11 is a diagram depicting an example of a template retained in atemplate retaining section.

FIG. 12 is a diagram depicting an example of a live distribution screenimage displayed on the outputting apparatus.

FIG. 13 is a diagram depicting an example of a message image.

FIG. 14 is a diagram depicting another example of a message image.

FIG. 15 is a diagram depicting an internal configuration of theinformation processing apparatus which operates as a reproductionapparatus of streaming data.

FIG. 16 is a diagram depicting a list screen image of contents.

FIG. 17 is a diagram depicting another example of a template retained inthe template retaining section.

FIG. 18 is a diagram depicting an example of a viewing screen imagedisplayed on the outputting apparatus.

FIG. 19 is a diagram illustrating a relationship between a viewerterminal and servers.

FIG. 20 is a diagram depicting an example of a game screen imagedisplayed on the outputting apparatus.

FIG. 21 is a diagram depicting another example of a viewing screen imagedisplayed on the outputting apparatus.

FIG. 22 is a diagram depicting an example of a command image.

DESCRIPTION OF EMBODIMENT

FIG. 1 depicts an information processing system 1 according to anembodiment of the present invention. The information processing system 1includes an information processing apparatus 10 of a distribution userof streaming data, a network server 5, sharing servers 9 a and 9 b, amanagement server 13, a store server 16, and information processingapparatuses 14 a and 14 b of viewing users of streaming data. Thecomponents mentioned are connected to each other through a network 3such as the Internet or a LAN (Local Area Network). The managementserver 13 manages session information of a game and the address (forexample, IP address) of the information processing apparatus 10 of thedistribution user and is illustrated here as a concept which supervisesa session server 11 and game servers 12 a and 12 b. The session server11 is maintained and managed by a management entity of the informationprocessing system 1, and the game servers 12 a and 12 b are maintainedand managed each by a maker, a publisher or the like of a game.

An access point (hereinafter referred to as “AP”) 8 has functions of awireless access point and a router, and the information processingapparatus 10 can connect to the AP 8 by wireless or wire connection soas to connect for communication to the network server 5, sharing servers9 a and 9 b, session server 11, game servers 12 a and 12 b and storeserver 16 on the network 3. Also the information processing apparatuses14 a and 14 b are similarly connected for communication with the networkserver 5, sharing servers 9 a and 9 b, session server 11, game servers12 a and 12 b and store server 16. It is to be noted that theinformation processing apparatus 10 and the information processingapparatuses 14 can be connected for communication directly without theintervention of a server such as the game servers 12.

The information processing apparatus 10 establishes connection to aninputting apparatus 6, which is operated by a user, by wireless or wireconnection, and the inputting apparatus 6 outputs operation informationrepresentative of a result of the operation of the user to theinformation processing apparatus 10. If the information processingapparatus 10 accepts operation information from the inputting apparatus6, then it reflects the operation information on processing of systemsoftware or application software, and a result of the processing isoutputted from an outputting apparatus 4. In the information processingsystem 1, the information processing apparatus 10 may be a gameapparatus which executes a game, and the inputting apparatus 6 may be anapparatus which supplies operation information of the user to theinformation processing apparatus 10 such as a game controller. In orderto play a game, the user would log in to OS (System Software) of theinformation processing apparatus 10. Users who log in to the systemsoftware are managed by user accounts registered in the informationprocessing apparatus 10.

The network server 5 is maintained and managed by the management entityof the information processing system 1 and provides a network service ofa game to users of the information processing system 1. The networkserver 5 manages network accounts for identifying the users, and a userwould use its network account to sign in to the network service providedby the network server 5. By signing in to the network service from theinformation processing apparatus 10, the user can register save data ofa game or a virtual award article (trophy) acquired during game playinto the network server 5.

The sharing servers 9 a and 9 b provide a service for sharing image datauploaded or distributed from the information processing apparatus 10and/or information relating to such image data. For example, the sharingserver 9 a may be a server which provides an SNS, and the sharing server9 b may be a server which provides an image sharing (distribution)service. In the following description, where the sharing servers 9 a and9 b are not distinguished from each other, each of them is referred toas “sharing server 9.” In the information processing system 1, thenumber of sharing servers 9 is not limited to two as depicted in FIG. 1but may be a greater number. In the information sharing service providedby the sharing servers 9, the users of interest may be limited toregistered members or may be open to the general. In the presentembodiment, “to share data or information” signifies to establish astate in which a user other than users who upload or distribute data orinformation to the sharing server 9 can access (for example, view) thedata or information, but does not make a question of whether or not auser who can access is limited to particular members.

It is to be noted that, in the present embodiment, the sharing server 9is configured as a streaming server which receives streaming datatransmitted thereto from the information processing apparatus 10,transcodes the streaming data and distributes the transcoded data as alive content. The information processing apparatus 10 encodes image datasame as image data which forms a display image of the outputtingapparatus 4 into data of a streaming format and transmits the encodeddata to the sharing server 9. The information processing apparatuses 14a and 14 b are connected to the sharing server 9 to receive thestreaming data and decodes and output the received data from outputtingapparatuses 15 a and 15 b, respectively. Here, the streaming data isimage data of a game played by a user A on the information processingapparatus 10, and the users of the information processing apparatuses 14a and 14 b can view a play video of the user A in a live state. In thefollowing description, where the information processing apparatuses 14 aand 14 b are not distinguished from each other, each of them is referredto as “information processing apparatus 14.”

The information processing apparatus 14 is a terminal apparatus having afunction of reproducing streaming data distributed from the sharingserver 9. Although the information processing apparatus 14 may be apersonal computer of the installation type or a mobile apparatus havinga Web browser function, it may be an apparatus same as the informationprocessing apparatus 10 of the distribution user, namely, a same gameapparatus. Where the information processing apparatus 14 of the viewinguser is the information processing apparatus 10 which incorporates aviewing application hereinafter described, the viewing user can enjoy asharing service which is high in convenience.

The management server 13 manages connection between the informationprocessing apparatus 14 and the information processing apparatus 10 whenthe user of the information processing apparatus 14 plays a gametogether with the user A of the information processing apparatus 10. Thegame servers 12 a and 12 b correspond to games different from each otherand individually generate sessions of the corresponding gamesindependently of each other. The form of a game session relies upon thegame, and conventionally it is necessary for a user who participates ina game session to directly access the game server 12 corresponding tothe game to apply for participation. In the information processingsystem 1 of the present embodiment, the session server 11 collectssession information from the game servers 12 a and 12 b andcomprehensively manages the session information. Therefore, a user whowants to participate can acquire the session information of thecorresponding game server 12 by accessing the session server 11.Consequently, in comparison with an alternative case in which the gameserver 12 is accessed directly, a simple and easy acquisition processcan be implemented.

In this manner, the information processing apparatus 14 acquires sessioninformation from the session server 11. If the information processingapparatus 14 is a game apparatus of a type same as that of theinformation processing apparatus 10, then the information processingapparatus 14 can acquire session information from the session server 11and can participate in a game being executed by the informationprocessing apparatus 10. The session server 11 may be maintained andmanaged by a management entity same as that of the network server 5.

It is to be noted that the session server 11 not only manages sessioninformation but also manages address information of the informationprocessing apparatus 10 connected to the network 3. The addressinformation of the information processing apparatus 10 may be, forexample, an IP address. For example, the session server 11 may beprovided with the address information of the information processingapparatus 10 from the network server 5. The store server 16 provides asite which sells an application of a game or the like.

The auxiliary storage apparatus 2 is a large capacity storage apparatussuch as an HDD (Hard Disk Drive) or a flash memory and may be anexternal storage apparatus connected to the information processingapparatus 10 by a USB (Universal Serial Bus) or may be a built-in typestorage apparatus. The outputting apparatus 4 may be a television sethaving a display unit which outputs an image and a speaker which outputssound or may be a computer display unit. The outputting apparatus 4 maybe connected to the information processing apparatus 10 by a wire cableor by wireless connection.

The inputting apparatus 6 is configured having a plurality of inputtingunits such as a plurality of operation buttons of the push type, ananalog stick which can input an analog quantity and turning buttons. Acamera 7 which is an image pickup apparatus is provided in the proximityof the outputting apparatus 4 and picks up an image of a space aroundthe outputting apparatus 4. While FIG. 1 depicts an example wherein thecamera 7 is attached to an upper portion of the outputting apparatus 4,the camera 7 may otherwise be disposed sidewardly of the outputtingapparatus 4. Whatever the case may be, the camera 7 is disposed at aposition at which the camera 7 can pick up an image of a user who playsa game in front of the outputting apparatus 4. The informationprocessing apparatus 10 has a function of authenticating the face of theuser from a picked up image of the camera 7.

FIG. 2 depicts an appearance configuration of the inputting apparatus 6.The user would grip a left side grip portion 78 b by the left hand andgrip a right side grip portion 78 a by the right hand to operate theinputting apparatus 6. A direction key 71, analog sticks 77 a and 77 band four kinds of operation buttons 76, which configure an inputtingunit, are provided on a housing upper face 79 of the inputting apparatus6. The four kinds of buttons 72 to 75 have different figures inscribedin different colors thereon in order to distinguish them from eachother. In particular, a red round mark is inscribed on the O button 72;a blue cross mark is inscribed on the X button 73; a violet square markis inscribed on the □ button 74; and a green triangle mark is inscribedon the Δ button 75. On the housing upper face 79, a touch pad may beprovided in a flat region between the direction key 71 and the operationbuttons 76.

A function button 80 is provided between the two analog sticks 77 a and77 b. The function button 80 is used to turn on the power supply to theinputting apparatus 6 and simultaneously make active a communicationfunction of connecting the inputting apparatus 6 and the informationprocessing apparatus 10 to each other. It is to be noted that, when themain power supply to the information processing apparatus 10 is in anoff state, if the function button 80 is depressed, the informationprocessing apparatus 10 accepts a connection request transmitted fromthe inputting apparatus 6 also as an instruction to turn on the mainpower supply. Consequently, the main power supply to the informationprocessing apparatus 10 is turned on. After the inputting apparatus 6 isconnected to the information processing apparatus 10, the functionbutton 80 is used also to cause the information processing apparatus 10to display a menu screen image.

A SHARE button 81 is provided on the left side of the left side analogstick 77 b. The SHARE button 81 is utilized to input an instruction fromthe user to the system software in the information processing apparatus10. An OPTION button 82 is provided on the right side of the right sideanalog stick 77 a. The OPTION button 82 is utilized to input aninstruction from the user to application (game) software executed by theinformation processing apparatus 10. The SHARE button 81 and the OPTIONbutton 82 may each be formed as a push-type button.

FIG. 3 depicts an appearance configuration of the inputting apparatus 6on the rear face side. A light emitting region 85 of a horizontallyelongated substantially rectangular shape is provided on a housing rearface 86 of the inputting apparatus 6. The light emitting region 85 hasLEDs of red (R), green (G) and blue (B), which are turned on inaccordance with emission light color information transmitted from theinformation processing apparatus 10. On the housing rear face 86, anupper side button 83 a, a lower side button 84 a and another upper sidebutton 83 b, another lower side button 84 b are provided at left-rightsymmetrical positions in a longitudinal direction. The upper side button83 a and the lower side button 84 a are operated by the forefinger andthe middle finger of the right hand of the user, respectively, and theupper side button 83 b and the lower side button 84 b are operated bythe forefinger and the middle finger of the left hand of the user,respectively. Since the light emitting region 85 is provided between therow of the upper side button 83 a and the lower side button 84 a on theright side and the row of the upper side button 83 b and the lower sidebutton 84 b on the left side as depicted in FIG. 3, the camera 7 cansuitably pick up an image of the light emitting region 85 that has beenturned on while the light emitting region 85 is not hidden by theforefinger or the middle finger by which a button is operated. The upperside buttons 83 may be configured as the push type buttons, and thelower side buttons 84 may be configured as the trigger type buttonswhich are rotatably supported.

FIG. 4 depicts an internal configuration of the inputting apparatus 6.The inputting apparatus 6 includes a wireless communication module 90, aprocessing part 91, the light emitting region 85 and an oscillator 98.The wireless communication module 90 has a function of transferring datato and from a wireless communication module of the informationprocessing apparatus 10. The processing part 91 executes expectedprocessing of the inputting apparatus 6.

The processing part 91 has a main controlling unit 92, an inputacceptance unit 93, a light emission controlling unit 94, a three-axisacceleration sensor 95, a three-axis gyro sensor 96 and a drivingcontrolling unit 97. The main controlling unit 92 carries out transferof necessary data to and from the wireless communication module 90.

The input acceptance unit 93 accepts operation information of aninputting unit such as the direction key 71, operation buttons 76,analog sticks 77, function button 80, SHARE button 81, OPTION button 82,upper side buttons 83 and lower side buttons 84 and transmits theoperation information to the main controlling unit 92. The maincontrolling unit 92 supplies the received operation information to thewireless communication module 90, and the wireless communication module90 transmits the operation information at a predetermined timing to theinformation processing apparatus 10. It is to be noted that the maincontrolling unit 92 may convert the received operation information intopredetermined control information as occasion demands.

The light emission controlling unit 94 controls emission of light of ared LED 87, a green LED 88 and a blue LED 89 which configure the lightemitting region 85. If the information processing apparatus 10 transmitsemission light color information for designating an emission light colorof the light emitting region 85, then the emission light colorinformation from the information processing apparatus 10 is received bythe wireless communication module 90 and is passed to the maincontrolling unit 92. The main controlling unit 92 notifies the lightemission controlling unit 94 of the emission light color information.Consequently, the light emission controlling unit 94 can control thelight emitting region 85 to emit light of the designated emission lightcolor.

The oscillator 98 is configured including an eccentric motor and isprovided in order to oscillate the entire inputting apparatus 6. Whenthe wireless communication module 90 receives an oscillation controllingsignal from the information processing apparatus 10, it supplies theoscillation controlling signal to the main controlling unit 92, and themain controlling unit 92 controls the driving controlling unit 97 tocause the oscillator 98 to oscillate in accordance with the oscillationcontrolling signal. The driving controlling unit 97 may be configured asa switch for driving the oscillator 98 or may be configured as a PWMcontrolling unit which varies the duty ratio of a supply voltage.

The three-axis acceleration sensor 95 detects acceleration components inthree-axis directions of XYZ of the inputting apparatus 6. Thethree-axis gyro sensor 96 detects angular velocities on an XZ plane, aZY plane and a YX plane. The main controlling unit 92 accepts detectionvalue information from the three-axis acceleration sensor 95 and thethree-axis gyro sensor 96, and the wireless communication module 90transmits the detection value information to the information processingapparatus 10 in a predetermined cycle together with operationinformation of a button input or the like.

FIG. 5 depicts a functional block diagram of the information processingapparatus 10. The information processing apparatus 10 is configuredincluding a main power supply button 20, a power ON LED 21, a standbyLED 22, a system controller 24, a clock 26, a device controller 30, amedia drive 32, a USB module 34, a flash memory 36, a wirelesscommunication module 38, a wire communication module 40, a sub system 50and a main system 60.

The main system 60 is configured having a main CPU (Central ProcessingUnit), and the sub system 50 is configured having a sub CPU. The mainCPU and the sub CPU operate exclusively. While the main CPU is startedand is in an active state, the sub CPU is in a standby state, but whilethe sub CPU is started and is in an active state, the main CPU is in astandby state. While the main power supply to the information processingapparatus 10 remains on, the main CPU executes various processes of anapplication or the like under an environment generated by systemsoftware while the sub CPU is asleep. On the other hand, if the mainpower supply to the information processing apparatus is turned off, thenthe main CPU starts up the sub CPU and enters an asleep state. In thismanner, the main CPU and the sub CPU operate exclusively.

The main power supply button 20 is an inputting unit to which anoperation input from the user is carried out and is provided on a frontface of a housing of the information processing apparatus 10 such thatit is operated in order to turn on or off the power supply to the mainsystem 60 of the information processing apparatus 10. In the followingdescription, that the main power supply is in an on state signifies thatthe main system 60 is in an active state, and that the main power supplyis in an off state signifies that the main system 60 is in a standbystate. The power ON LED 21 is turned on when the main power supplybutton 20 is turned on, and the standby LED 22 is turned on when themain power supply button 20 is turned off. It is to be noted that such aconfiguration that turning on/off of the main power supply is carriedout by a short press operation of the main power supply button 20 but,if the main power supply button 20 is long-pressed, then the systempower supply to the information processing apparatus 10 is turned off tostop the power supply to the information processing apparatus 10 may beadopted.

The system controller 24 detects depression of the main power supplybutton 20 by the user. If the main power supply button 20 is depressedwhile the main power supply is in an off state, then the systemcontroller 24 acquires the depression operation as an “on instruction,”but if the main power supply button 20 is depressed while the main powersupply is in an on state, then the system controller 24 acquires thedepression operation as an “off instruction.” It is to be noted thatturning on/off of the main power supply can be controlled from theinputting apparatus 6 as described hereinabove, and if the functionbutton 80 of the inputting apparatus 6 is depressed when the main powersupply is in an off state, then the system controller 24 acquires thebutton operation as an “on instruction.”

If the system controller 24 acquires an on instruction, then it notifiesthe sub system 50, which is in an active state, of a result of thedetection and turns off the standby LED 22 while it turns on the powerON LED 21. At this time, the sub system 50 starts up the main system 60and itself enters a standby mode. On the other hand, if the systemcontroller 24 acquires an off instruction, then it notifies the mainsystem 60, which is in an active state, of a result of the detection andturns off the power ON LED 21 while it turns on the standby LED 22. Atthis time, the main system 60 starts up the sub system 50 and itselfenters a standby mode.

While the main CPU has a function of executing a game program installedin the auxiliary storage apparatus 2, the sub CPU does not have thefunction. However, the sub CPU has a function of accessing the auxiliarystorage apparatus 2 and another function of transferring information toand from the network server 5. The sub CPU is configured having onlysuch limited processing functions and accordingly can operate with lowerpower consumption than the main CPU.

The clock 26 is a real time clock, and generates date and timeinformation at present and supplies the generated information to thesystem controller 24, sub system 50 and main system 60.

The device controller 30 is configured as an LSI (Large-Scale IntegratedCircuit) which executes delivery of information between devices like asouth bridge. As depicted in FIG. 5, such devices as the systemcontroller 24, media drive 32, USB module 34, flash memory 36, wirelesscommunication module 38, wire communication module 40, sub system 50 andmain system 60 are connected to the device controller 30. The devicecontroller 30 absorbs a difference in electric characteristic, adifference in data transfer rate between the devices and controls thetiming of data transfer.

The media drive 32 is a drive apparatus which operates a ROM medium 44,on which application software of a game or the like is recorded, loadedthereon to read out a program, data and so forth from the ROM medium 44.The ROM medium 44 is a read-only recording medium such as an opticaldisk, a magneto-optical disk or a Blu-ray disk.

The USB module 34 is a module to be connected to an external apparatusby a USB cable. The USB module 34 may be connected to the auxiliarystorage apparatus 2 and the camera 7 by a USB cable. The flash memory 36is an auxiliary storage apparatus which configures an internal storage.The wireless communication module 38 wirelessly communicates, forexample, with the inputting apparatus 6 using a communication protocolsuch as a Bluetooth (registered trademark) protocol or an IEEE 802.11protocol. It is to be noted that the wireless communication module 38may be compatible with a third generation (3rd Generation) digitalportable telephone system compliant with the IMT-2000 (InternationalMobile Telecommunication 2000) standard prescribed by the ITU(International Telecommunication Union). The wire communication module40 wire-communicates with an external apparatus and is connected to thenetwork 3, for example, through the AP 8.

The main system 60 includes a main CPU, a memory which is a main storageapparatus and a memory controller, a GPU (Graphics Processing Unit) andso forth. The GPU is utilized principally for arithmetic processing of agame program. Those functions may be configured as a system-on-chip andformed on one chip. The main CPU has a function of starting up systemsoftware and executing an application installed in the auxiliary storageapparatus 2 under an environment provided by the system software.

The sub system 50 includes a sub CPU, a memory which is a main storageapparatus, a memory controller and so forth but does not include a GPU.The number of circuit gates of the sub CPU is smaller than the number ofcircuit gates of the main CPU, and the operation power consumption ofthe sub CPU is lower than that of the main CPU. As describedhereinabove, the sub CPU operates while the main CPU is in a standbystate, and the processing functions of the sub CPU are limited in orderto suppress the power consumption of the sub CPU low. It is to be notedthat the sub CPU and the memory may be formed on chips separate fromeach other.

FIG. 6 depicts an internal configuration of the information processingapparatus 10 which operates as a distribution apparatus of streamingdata. The information processing apparatus 10 includes a processing part100, a communication unit 102 and an acceptance unit 104. The processingpart 100 includes an application execution unit 106, an image generationunit 110, an image changeover unit 130 and a sharing processing unit150.

The image generation unit 110 generates image data to be displayed onthe outputting apparatus 4 and includes a game image generation section112, a picked up image generation section 114, a comment imagegeneration section 116, a status image generation section 118 and a menuimage generation section 120. Each of the game image generation section112, picked up image generation section 114, comment image generationsection 116 and status image generation section 118 generates image datato be displayed in a live distribution layout and displays a livedistribution image on the outputting apparatus 4. The menu imagegeneration section 120 generates menu image data in which icon images ofgames or applications are arrayed and displays the menu image data onthe outputting apparatus 4. The image changeover unit 130 changes overan image to be displayed on the outputting apparatus 4 between a menuimage and a live distribution image.

The sharing processing unit 150 carries out a process of transmittinglive distribution image data generated by the image generation unit 110to the sharing server 9 so that a live distribution image is shared witha different user. The sharing processing unit 150 includes an inputimage generation section 152, a confirmation processing section 154, anencoding section 156, a distribution processing section 158, a templateretaining section 160 and an information acquisition section 162.

The components represented as functional blocks which carry out variousprocesses in FIG. 6 can be configured, in hardware, from a circuitblock, a memory and other LSIs, and is implemented, in software, by aprogram and so forth loaded in the memory. Accordingly, it can berecognized by those skilled in the art that the functional blocks can beimplemented in various forms only from hardware, only from software orfrom a combination of hardware and software and are not limited to anyof them.

In the information processing system 1 of the present embodiment, thecommunication unit 102 receives information that an inputting unit ofthe inputting apparatus 6 is operated by a user (such information ishereinafter referred to as “operation information”). Further, thecommunication unit 102 transmits a content generated by the processingpart 100 to the sharing server 9. Here, the content is a livedistribution image generated by the image generation unit 110. Thecommunication unit 102 is represented as a configuration having both offunctions of the wireless communication module 38 and the wirecommunication module 40 depicted in FIG. 5.

The acceptance unit 104 is provided between the communication unit 102and the processing part 100 and transmits data or information to andfrom the communication unit 102 and the processing part 100. If theacceptance unit 104 accepts operation information of an inputting unitprovided in the inputting apparatus 6 through the communication unit102, then it supplies the operation information to one, two or more ofthe application execution unit 106, image generation unit 110, imagechangeover unit 130, and sharing processing unit 150 of the processingpart 100.

If the main power supply button 20 is depressed by the user in a statein which the main power supply to the information processing apparatus10 is off, then the main system is started up. The main system 60supplies a generation instruction of a menu image to the menu imagegeneration section 120, and the menu image generation section 120generates a menu image in which icon images of applications and the likeare arrayed and outputs the menu image to the outputting apparatus 4. Itis to be noted that the menu image generation section 120 may generate amenu image when the user depresses the function button 80 of theinputting apparatus 6 in a state in which the main power supply to theinformation processing apparatus 10 is on. The function of the menuimage generation section 120 is implemented by system software.

FIG. 7 depicts an example of a menu screen image displayed on theoutputting apparatus 4. The menu screen image is displayed when the useris to select a game to be played or when the user browses activities ofother users. In a category selection region 204, categories of menusprepared in the information processing apparatus 10 are displayed. In agame icon display region 200, icons of games which can be played by alog-in user and icons of applications which can be executed aredisplayed in a list in a horizontal direction of the screen. The usercan operate the inputting apparatus 6 to scroll the list rightwardly orleftwardly. In the game icon display region 200, a focus region 210 isset at a predetermined position, and if a game icon in which the user isinterested is placed into the focus region 210, then icons relating tothe game are displayed in a relating icon display region 202 andrelating information is displayed in relating information displayregions 206 and 208. If the user operates an inputting unit of theinputting apparatus 6 to place the icon of a game to be played into thefocus region 210 and then depresses a predetermined determination key,then the acceptance unit 104 depicted in FIG. 6 accepts the operationinformation as a game execution command. Thus, the application executionunit 106 executes the game.

In the relating information display region 206, relating information tothe game placed in the focus region 210 is displayed, and, for example,a moving picture relating to the game is displayed. The moving picturedisplayed may be a play moving picture of the game accumulated in thesharing server 9 or a live moving picture distributed from the sharingserver 9. In the relating information display region 208, relatinginformation to the game placed in the focus region 210 is displayed,and, for example, a list of friends who are playing the game may bedisplayed. The menu image generation section 120 generates such a menuscreen image and displays the menu screen image from the outputtingapparatus 4.

If an execution command of a game is inputted by the user, then theapplication execution unit 106 executes a program of the game placed inthe focus region 210. In the present example, a game of the title“SAMURAI” is started.

While the application execution unit 106 in the present embodimentexecutes a game program, it may otherwise execute a different kind of anapplication program. The application execution unit 106 carries outarithmetic operation processing for moving a game character in a virtualspace based on operation information inputted to the inputting apparatus6 from the user. Here, the application execution unit 106 may be anapplication (game program) itself. The game image generation section 112may be a GPU (Graphics Processing Unit) which executes a renderingprocess or the like, and receives a result of processing by theapplication execution unit 106 and generates image data of theapplication (game) to be displayed on the outputting apparatus 4. FIG. 8depicts an example of a game screen image displayed on the outputtingapparatus 4.

The sharing processing unit 150 in the present embodiment provides amechanism for sharing game image data generated by the image generationunit 110 with other users. This sharing process is executed taking it asa trigger that a particular inputting unit provided on the inputtingapparatus 6 is operated by the user.

Referring back to FIGS. 2 and 3, a plurality of inputting units providedon the inputting apparatus 6 can be classified into three types from apoint of view of inputting of operation information to the informationprocessing apparatus 10.

(First Type)

An inputting unit of the first type is configured to input operationinformation to the system software of the information processingapparatus 10, and particularly, the function button 80 and the SHAREbutton 81 belong to the first type. Operation information of thefunction button 80 and the SHARE button 81 is provided only to thesystem software but is not provided to the application software. Here,the system software signifies software for managing/controlling thehardware of the computer of the information processing apparatus 10 tosupport work or use of the information processing apparatus 10.Meanwhile, the application software signifies software which directlyhas a function of carrying out a work to be executed by the user in anenvironment constructed by the system software. Representatively, thesystem software includes the OS, and the application software includesgame software.

(Second Type)

An inputting unit of the second type is configured to input operationinformation to the application software of the information processingapparatus 10, and particularly, the OPTION button 82 belongs to thesecond type. Operation information of the OPTION button 82 is providedonly to the application software but is not provided to the systemsoftware.

(Third Type)

An inputting unit of the third type is configured to input operationinformation to any of the system software and the application softwareof the information processing apparatus 10, and particularly, thedirection key 71, operation buttons 76, analog sticks 77, upper sidebuttons 83 and lower side buttons 84 belong to the third type. Duringdisplay of a menu screen image (refer to FIG. 7) generated by the systemsoftware, operation information of an inputting unit of the third typeis provided to the system software. However, during display of a gamescreen image (refer to FIG. 8) generated by the application software,operation information of an inputting unit of the third type is providedto the application software. In this manner, operation information of aninputting unit of the third type is provided to a generation entity of ascreen image displayed on the outputting apparatus 4.

If the acceptance unit 104 accepts operation information of an inputtingunit of the first type, particularly, of the SHARE button 81, from amongthe inputting units of the three types, then the sharing processing unit150 carries out a process of sharing image data generated by the imagegeneration unit 110 or information relating to the image data with adifferent user. This sharing process is not carried out when theacceptance unit 104 accepts operation information of an inputting unitof the second type or the third type, but is carried out absolutely onlywhen operation information of an inputting unit of the first type isaccepted.

In the following, a sharing process when the user operates the SHAREbutton 81 during display of the game screen image depicted in FIG. 8 isdescribed.

The input image generation section 152 generates an inputting imageindicative of choices relating to sharing of image data. On the inputscreen image displayed on the outputting apparatus 4, a GUI (GraphicalUser Interface) for inputting predetermined information through anoperation of the inputting apparatus 6 by the user, a text inputtingfield and so forth are disposed. In the following description, dataprovided to the sharing server 9 is sometimes referred to as “content.”On the inputting image, a GUI for selecting a content which is to makean object of a sharing process is displayed.

FIG. 9 depicts an example of an input screen image displayed in anoverlapping relationship on a game screen image. On the input screenimage 220, three choices relating to content sharing are presented.“Upload Screenshot” is a GUI which designates uploading of image data ofa screenshot; “Upload Video” is a GUI which designates uploading ofimage data recorded in a ring buffer in the information processingapparatus 10; and “Video Streaming” is a GUI which designates liverelaying of a play video of the game. The user would operate theinputting apparatus 6 to select one of the GUIs and depress the Enterkey to execute the selected sharing process.

FIG. 10 depicts an example of an input screen image displayed in anoverlapping relationship on a game screen image when the “VideoStreaming” is selected. On the input screen image 222, choices relatingto sharing of a content are indicated, and here, a service selectionfield 230 for inputting information regarding which one of contentsharing services provided by a plurality of sharing servers 9 is to beselected is indicated. In this example, two choices of a distributionservice A and another distribution service B are presented. Although thedistribution services may be operated by different business entities,they may otherwise be operated by a single business entity, by whichdifferent kinds of services may be provided.

The user can input a comment relating to a live distribution image inthe form of a text. The inputted comment may be displayed together witha thumbnail image of the content on the sharing site. Since it isexpected that the interest of a different user in the content is raisedby browsing the comment displayed on the sharing site, inputting of thecommand has important implications in the sharing service. It is to benoted that the inputted comment may be transmitted not only to a site ofa service selected through the service selection field 230 but also to asite, for example, of an SNS so that it can be browsed by other userstogether with information of the URL or the like of the sharing servicewhich carries out content distribution. This makes it possible for theother users to recognize that a content has been distributed and makesit possible to access the sharing server 9 readily.

A picture quality selection field 226 indicates choices for selectingpicture quality of streaming data to be distributed. In the presentexample, three kinds of picture quality including “high picturequality,” “medium picture quality” and “low picture quality” arepresented. However, “highest picture quality” may be added to the kindsof picture quality so that picture quality can be selected from amongthe four kinds of picture quality. If the user selects a distributionservice and distribution picture quality using the inputting apparatus 6and inputs a comment and then selects a distribution starting button224, then the distribution processing section 158 carries out aconnection process to the sharing server 9 which provides the selecteddistribution service. It is to be noted that, while the inputting screenimage depicted in FIG. 10 is configured such that a distributiondestination service and distribution picture quality can be selected anda comment can be inputted, choices regarding whether or not microphonesound or a picked up image of the camera 7 is to be included indistribution or regarding whether or not a comment from a viewing useris to be displayed may be included additionally.

If connection between the information processing apparatus 10 and thesharing server 9 is established, then the image generation unit 110generates a live distribution image in accordance with the templateretained in the template retaining section 160. The live distributionimage is outputted to and displayed on the outputting apparatus 4 and isencoded by the sharing processing unit 150 and then distributed to thesharing server 9. It is to be noted that, although, in the informationprocessing system 1 of the present embodiment, a live distribution imageis distributed to the information processing apparatus 14 of the viewerthrough the sharing server 9, the information processing apparatus 10and the information processing apparatus 14 may establish connectiontherebetween so that a live distribution image is distributed from theinformation processing apparatus 10 to the information processingapparatus 14 without the intervention of the sharing server 9.

FIG. 11 depicts an example of the template retained in the templateretaining section 160. A template 232 defines a display layout of theentire display of the outputting apparatus 4. In an image display region234, a game image is displayed. In a status display region 236, adistribution status is displayed. In a picked up image display region238, a picked up image of the camera 7 is displayed. In a commentdisplay region 240, a comment from a different user is displayed. Theimage generation unit 110 generates images in the display regions togenerate live distribution image data. It is to be noted that thetemplate 232 depicted in FIG. 11 is an example, and a templateequivalent to the template 232 from which the comment display region 240is excluded, another template equivalent to the template 232 from whichthe picked up image display region 238 is excluded, a further templateequivalent to the template 232 from which the picked up image displayregion 238 and the comment display region 240 are excluded and someother template may be retained in the template retaining section 160.The image generation unit 110 reads out a template to be used from thetemplate retaining section 160 in accordance with information inputtedthrough the input screen image 222 depicted in FIG. 10.

FIG. 12 depicts an example of a live distribution screen image displayedon the outputting apparatus 4. The image generation unit 110 generateslive distribution image data including a game image in accordance withthe template 232 and outputs the live distribution image data to theoutputting apparatus 4 and the sharing processing unit 150. The encodingsection 156 encodes the live distribution image data into data of astreaming format, and the distribution processing section 158 transmitsthe encoded streaming data to the sharing server 9. This makes itpossible for the information processing apparatus 14 of a viewing userto access the sharing server 9 and display a live distribution imageprovided from the information processing apparatus 10 on the outputtingapparatus 15. The sharing server 9 manages the viewing situation of thelive distribution image and notifies the information processingapparatus 10 of the viewing situation. Further, the sharing server 9transmits a comment transmitted thereto from the information processingapparatus 14 to the information processing apparatus 10. In theinformation processing apparatus 10, the information acquisition section162 acquires information relating to the live distribution image datasuch as the viewing situation and the comment.

The game image generation section 112 generates game image data, and agame image is displayed in the image display region 234. The picked upimage generation section 114 acquires a camera image from the camera 7and generates picked up image data, and a picked up image is displayedin the picked up image display region 238. The comment image generationsection 116 generates comment image data using comment data acquired bythe information acquisition section 162, and a comment image isdisplayed in the comment display region 240. It is to be noted that thecomment image generation section 116 may generate a comment image fromone viewing user and successively generate comment image data withreference to a minimum display time period for a comment. For example,if the minimum display time period for a comment is set to three secondsand the information acquisition section 162 acquires a comment from thesharing server 9 after every predetermined interval of time (forexample, once per one minute), then if a plurality of comments areacquired, then the comment image generation section 116 successivelygenerates comment images for one minute. Consequently, new commentimages are successively displayed in the comment display region 240. Itis to be noted that, if the number of acquired comments is equal to orgreater than 21, then if one comment is displayed for three seconds,then some comment cannot be displayed, and therefore, a comment orcomments which cannot be displayed may be discarded. The status imagegeneration section 118 generates status image data using a viewingsituation acquired by the information acquisition section 162, and astatus image is displayed in the status display region 236. It is to benoted that also the number of comments inputted from viewers may bedisplayed in the status display region 236. In this manner, the imagegeneration unit 110 generates live distribution image data in accordancewith the template 232.

The live distribution image at least includes a game image and images ofinformation acquired by the information acquisition section 162. In thepresent example, as images of information acquired by the informationacquisition section 162, a status image in the status display region 236and a comment image in the comment display region 240 are included. Apicked up image of the camera 7 may be or may not be included in thelive distribution image depending upon a desire of the user. Theencoding section 156 encodes live distribution image data generated bythe image generation unit 110, and the distribution processing section158 transmits the encoded live distribution image data to the sharingserver 9 so that a viewer of the live distribution image can view acomment image or a status image same as that viewed by the user A who isthe distributor. It is to be noted that, where a microphone is connectedto the information processing apparatus 10, the distribution processingsection 158 transmits also sound from the microphone to the sharingserver 9 in synchronism with the live distribution image. Consequently,if the user A mentions an impression on the comment displayed in thecomment display region 240, then also the voice of the impression isconveyed to the viewer. Accordingly, an environment in which it seemsthat the viewer and the distributor (user A) exist in the same space isimplemented.

It is to be noted that a game sometimes has embedded therein informationwhich inhibits recording or distribution thereof in order to preventrevelation of the contents. For example, at a high point of a game suchas a special stage or a fight scene against a boss character,information for inhibition of recording/distribution is set. In thefollowing description, a scene or a stage at whichrecording/distribution is inhibited is referred to as “inhibitioninterval.” The confirmation processing section 154 carries outconfirmation of whether or not a game image may be distributed. Ifinhibition information is received from the game, then the confirmationprocessing section 154 confirms that distribution of game image datawithin the inhibition interval is not permitted and notifies thedistribution processing section 158 of this. In response to thenotification, the distribution processing section 158 stops distributionof the live distribution image and distributes a message image 250depicted in FIG. 13 to the sharing server 9. In particular, the encodingsection 156 stops encoding of a live distribution image and encodes theprescribed message image 250, and the distribution processing section158 distributes the encoded message image. Consequently, on theoutputting apparatus 4, the live distribution image depicted in FIG. 12is displayed. Meanwhile, on the outputting apparatus 15 of the viewer,the message image 250 is displayed. If the confirmation processingsection 154 detects an end of the inhibition interval, then thedistribution processing section 158 resumes distribution of a livedistribution image. It is to be noted that, when the distributionprocessing section 158 confirms inhibition of distribution of game imagedata, the image generation unit 110 may overwrite the game image data inthe live distribution image data with the message image 250 such thatthe live distribution image data in which the game image is replacedwith the message image 250 is distributed from the distributionprocessing section 158. It is to be noted that, within the inhibitioninterval, preferably the confirmation processing section 154 displaysinformation representing the inhibition interval in the livedistribution image displayed on the outputting apparatus 4 to notify thedistribution user of the information.

If the user A depresses the function button 80 during distribution of alive distribution image, then the image changeover unit 130 changes overthe image to be displayed on the outputting apparatus 4 to the menuimage (refer to FIG. 7). It is to be noted that, if the user A depressesa predetermined button during display of the menu image, then the imagechangeover unit 130 changes over the image to be displayed on theoutputting apparatus 4 to the live distribution image. In this manner,the image changeover unit 130 has a function of changing over the imageto be displayed on the outputting apparatus 4 between the menu image andthe game image. It is to be noted that, also when the image changeoverunit 130 changes over the live distribution image to the menu image, ifthe application execution unit 106 continues the game, then the gameimage generation section 112 may continue to generate a game image onthe background.

Although the distribution processing section 158 distributes a livedistribution image including a game image to the sharing server 9, itdoes not distribute the menu image. In other words, when the menu imageis displayed on the outputting apparatus 4, the distribution processingsection 158 transmits image data which does not include the menu imageto the sharing server 9. For example, FIG. 14 depicts a message image252 which is transmitted to the sharing server 9 during display of themenu image, and the viewer will see a message “Please wait a little”until distribution of a live distribution image is resumed.

A configuration and operation of the information processing apparatus 10which serves as a distribution source of streaming data have beendescribed. Now, a configuration and operation of the informationprocessing apparatus 14 which accesses the sharing server 9 to reproducestreaming data are described. It is to be noted that, although theinformation processing apparatus 14 may be a terminal apparatus whichhas a Web browser function such as a personal computer or a mobileterminal as described hereinabove, the following description is given ofa case in which the information processing apparatus 14 is theinformation processing apparatus 10 which has the functional blocksdepicted in FIG. 5. It is to be noted that a viewing user providesoperation information to the information processing apparatus 14 usingthe inputting apparatus 6. In this manner, in the information processingsystem 1, the information processing apparatus 10 has a function that itoperates as a distribution apparatus which distributes streaming dataand operates also as a reproduction apparatus which reproduces streamingdata.

FIG. 15 depicts an internal configuration of the information processingapparatus 10 which operates as a reproduction apparatus of streamingdata. The information processing apparatus 10 includes a processing part100, a communication unit 102 and an acceptance unit 104. The processingpart 100 includes an application execution unit 106, an acquisition unit300, an image generation unit 320, a command processing unit 340, acomment processing unit 360 and a template retaining unit 370. It is tobe noted that, when the information processing apparatus 10 operates asa reproduction apparatus, the application execution unit 106 starts up aviewing application of a live distribution image. As the viewingapplication is started up, the functions of the acquisition unit 300,image generation unit 320, command processing unit 340 and commentprocessing unit 360 are implemented.

The acquisition unit 300 acquires various kinds of information and datafrom various servers existing on the network 3. The acquisition unit 300includes a streaming data acquisition section 302, a specificationinformation acquisition section 304, a command information acquisitionsection 306, a store information acquisition section 308, a sessioninformation acquisition section 310, a comment data acquisition section312 and a content information acquisition section 314. The imagegeneration unit 320 generates image data for being displayed on theoutputting apparatus 15 and includes a streaming data reproductionsection 322, a command image generation section 324, a purchase markgeneration section 326, a participation mark generation section 328, acomment image generation section 330 and a list image generation section332. The command processing unit 340 includes a command input acceptancesection 342 and a command transmission section 344, and the commentprocessing unit 360 includes a comment input acceptance section 362 anda comment transmission section 364.

The components represented as functional blocks which carry out variousprocesses in FIG. 15 can be configured, in hardware, from a circuitblock, a memory and other LSIs, and is implemented, in software, by aprogram and so forth loaded in the memory. Accordingly, it can berecognized by those skilled in the art that the functional blocks can beimplemented in various forms only from hardware, only from software orfrom a combination of hardware and software and are not limited to anyof them.

The communication unit 102 receives data and information transmittedfrom the servers and receives operation information representative of anoperation of an inputting unit of the inputting apparatus 6 by the user.Further, the communication unit 102 transmits a command or a commentgenerated by the processing part 100 to an arbitrary server. Thecommunication unit 102 is represented as a configuration which has bothof functions of the wireless communication module 38 and the wirecommunication module 40 depicted in FIG. 5.

The acceptance unit 104 is provided between the communication unit 102and the processing part 100 and transmits data or information to andfrom the communication unit 102 and the processing part 100. If theacceptance unit 104 accepts operation information of an inputting unitprovided in the inputting apparatus 6 through the communication unit102, then it supplies the operation information to one, two or more ofthe acquisition unit 300, image generation unit 320, command processingunit 340, comment processing unit 360 and application execution unit 106of the processing part 100.

In the menu screen image depicted in FIG. 7, an icon of a viewingapplication of a live distribution image is placed in the focus region210, and if the user depresses the predetermined determination key ofthe inputting apparatus 6, then the acceptance unit 104 depicted in FIG.15 accepts the operation information as an application startup commandand the application execution unit 106 executes the viewing application.When the viewing application is executed, the functions of theacquisition unit 300, image generation unit 320, command processing unit340 and comment processing unit 360 are implemented. When the viewingapplication is started up, the information processing apparatus 10establishes connection to the sharing server 9, and the image generationunit 320 generates a list of contents streaming distributed from thesharing server 9. It is to be noted that the connection between theinformation processing apparatus 10 and the sharing server 9 may becarried out by the viewing application or may be carried out manually bythe user.

FIG. 16 depicts a list screen image of contents. When the viewingapplication is started up, the content information acquisition section314 acquires information of contents being live distributed from thesharing server 9. The information of contents includes, for example,thumbnail images of application videos being distributed, informationfor specifying distributors, information for specifying applications andcomment data and may be information registered as attribute informationof the contents in the sharing server 9. The list image generationsection 332 uses the content information acquired by the contentinformation acquisition section 314 to generate a list screen image inwhich the thumbnail images, distributor names and game titles arejuxtaposed. The user would operate the inputting apparatus 6 to move afocus frame 290 to place the focus frame 290 at a thumbnail image to beviewed. In the example depicted in FIG. 16, the focus frame 290 isplaced at the thumbnail image of the game title “SAMURAI” which is beingdistributed by the user A.

The streaming data acquisition section 302 transmits information forspecifying the content at which the focus frame 290 is placed to thesharing server 9, and the sharing server 9 distributes streaming dataspecified by the content specification information to the informationprocessing apparatus 10. The streaming data reproduction section 322decodes and reproduces the streaming data, and consequently, a livevideo is displayed in the region in which the focus frame 290 is placed.By moving and placing the focus frame 290 to and at a thumbnail image,the user can view the live video of the content corresponding to thethumbnail image. It is to be noted that, after the focus frame 290 isplaced at a thumbnail image, the list image generation section 332 maydisplay information of a comment or the like in the proximity of thefocus frame 290. If the user depresses the Enter key of the inputtingapparatus 6, a viewing screen image of the content at which the focusframe 290 is placed is displayed. The image generation unit 320generates viewing image data in accordance with a template retained inthe template retaining unit 370 and outputs the viewing image data tothe outputting apparatus 15.

FIG. 17 depicts an example of a template retained in the templateretaining unit 370. A template 272 defines a display layout of theentire display of the outputting apparatus 15. In a live video displayregion 274, a live video is displayed. In a command image display region276, a command image for inputting a command to an application beingplayed by the distributor is displayed. In a participation mark displayregion 278, a participation mark for inputting a participation requestto the application being played by the distributor is displayed. In apurchase mark display region 280, a purchase mark for inputting apurchase request to the application being played by the distributor isdisplayed. A comment input field 284 is a region into which the viewinguser is to input a comment in the form of a text, and if a comment markdisplayed in a comment mark display region 282 is selected, then acomment inputted to the comment input field 284 is transmitted to theinformation processing apparatus 10 of the distribution user A throughthe sharing server 9. A comment display region 286 is a region in whicha comment from a viewing user who is viewing the content is displayed.

FIG. 18 depicts an example of a viewing screen image displayed on theoutputting apparatus 15. The image generation unit 320 generates viewingimage data including a live video in accordance with the template 272and outputs the viewing image data to the outputting apparatus 15. Thestreaming data acquisition section 302 acquires streaming data includinga game image distributed from the sharing server 9. The streaming datareproduction section 322 decodes the acquired streaming data toreproduce a live video, and the reproduced live video is displayed onthe live video display region 274.

The specification information acquisition section 304 acquiresspecification information of a distributor who provides streaming dataand specification information of an application. It is to be noted thatthe specification information acquisition section 304 may be replaced bythe content information acquisition section 314 described hereinabove ormay receive distributor specification information of a content to beviewed and application specification information from the contentinformation acquisition section 314.

The command information acquisition section 306 acquires commandinformation to an application based on application specificationinformation. The command information is data necessary to displayinformation (image) for specifying a command in the command imagedisplay region 276 and may be, for example, a widget for displaying aGUI button. The game “SAMURAI” being executed in the informationprocessing apparatus is configured such that a command from an externaluser is accepted and reflected on processing of the application. Here,the external user need not have “SAMURAI” installed in the informationprocessing apparatus 10 thereof, and in short, only if the external useris viewing a live video distributed from the sharing server 9, it cantransmit a command to the game “SAMURAI.” The viewing user need notnecessarily utilize the information processing apparatus 10 which is agame apparatus, but preferably, also a user of a tablet terminal havinga Web browser can transmit a command as hereinafter described.

The store information acquisition section 308 acquires informationrelating to purchase of an application based on applicationspecification information. The information relating to purchase of anapplication may be an address (URI: Uniform Resource Identifier) on thestore server 16 from which the game “SAMURAI” is sold.

The session information acquisition section 310 acquires sessioninformation based on distributor specification information. Here, thesession information is generated when a distribution user permitsparticipation of a different user into a game being played, and whereparticipation is not permitted, no session information is generated. Inthe information processing system 1, when a distribution user permitsparticipation of a different user, the game server 12 issues a sessionID for specifying a session of the game. The session ID may be a URI forconnection to the game server 12 which is utilized by the informationprocessing apparatus 10 which is to participate in a session to carryout synchronous communication through the game server 12. Sessioninformation including an issued session ID and distribution userspecification information is passed to the session server 11. Thesession server 11 accepts and manages session information from aplurality of game servers 12. Accordingly, the session informationacquisition section 310 accesses the session server 11 to confirmwhether the game being played by the distribution user permitsparticipation, and where participation is permitted, the sessioninformation acquisition section 310 acquires session information.

The information acquisition process is described. FIG. 1 s a diagramillustrating a relationship between a viewer terminal and servers.

Where the distribution user permits participation in the game “SAMURAI,”the information processing apparatus 10 notifies the game server 12 ofpermission of participation. The game server generates and transmits asession ID to the information processing apparatus 10 and transmits alsoto the session server which manages session information in theinformation processing system 1 in a concentrated manner (S10).Consequently, the distribution user can accept a participation requestfrom other information processing apparatus 10 and can play the gamewith a different user. It is to be noted that the information processingapparatus 10 of the distribution user distributes encoded streaming datato the sharing server 9.

The viewing user starts up a viewing application in the informationprocessing apparatus 10 (S12). The information processing apparatus 10establishes connection to the sharing server 9 (S14), and the sharingserver 9 transmits the distributor specification information and thegame specification information to the information processing apparatus10 (S16). It is to be noted that the distributor specificationinformation and the game specification information may be transmittedwhen a content list screen image is generated in the informationprocessing apparatus 10 or may be transmitted when a content is selectedfrom within a list screen image by a viewing user.

The command information acquisition section 306 issues a request forcommand information to the game server 12 specified by the gamespecification information (S18). As described hereinabove, the gameserver 12 is a server which processes a session of a game and isprovided for each game in the information processing system 1.Accordingly, the command information acquisition section 306 specifiesthe game server 12 which processes a session of the game “SAMURAI” fromthe game specification information and issues a request for commandinformation to the specified game server 12. Here, if “SAMURAI” is agame of the type which does not accept a command from an external user,the game server 12 notifies the information processing apparatus 10 thatno command information exists. On the other hand, if “SAMURAI” is a gameof the type which accepts a command from an external user, then the gameserver 12 transmits command information to the information processingapparatus 10. The command information acquisition section 306 acquiresthe command information and passes the command information to thecommand image generation section 324.

The session information acquisition section 310 confirms presence orabsence of session information to the session server 11 based on thegame specification information (S22). If the distribution user does notpermit participation in the game, then no session information exists. Onthe other hand, if the distribution user permits participation in thegame, then the session server 11 transmits the session information tothe information processing apparatus 10 (S24). The session informationacquisition section 310 acquires the session information and passes thesession information to the participation mark generation section 328.

The store information acquisition section 308 issues a request for anaddress which provides a sale screen image of the game to the storeserver 16 based on the game specification information (S26). The storeserver 16 transmits the address of the sale page of the game “SAMURAI,”and the store information acquisition section 308 acquires the addressand passes the address to the purchase mark generation section 326.

It is to be noted that, after the establishment of connection at S14,though not depicted, live distribution of streaming data andtransmission of comment data are carried out from the sharing server 9to the information processing apparatus 10 and are acquired by thestreaming data acquisition section 302 and the comment data acquisitionsection 312, respectively.

It is to be noted that the session server 11 may manage user informationand address information of the information processing apparatus 10connected to the network 3 and information for specifying a game beingexecuted in the information processing apparatus 10 irrespective ofwhether or not a session exists. In this case, if the acquisition unit300 of the information processing apparatus 14 transmits distributorspecification information and game specification information of acontent to the session server 11, then the session server 11 confirmsthe user and the user specification information of the informationprocessing apparatus 10. If the session server 11 has commandinformation to the game retained therein in advance, then the sessionserver 11 transmits the command information to the informationprocessing apparatus 14, but if the session server 11 has the address ofthe sale page of the game retained therein in advance, then the sessionserver 11 transmits the address to the information processing apparatus14. It is to be noted that, if the session server 11 does not have thecommand information retained therein, then it acquires the commandinformation from the corresponding game server 12. Further, if thesession server 11 does not have the address of the sale page retainedtherein, then it acquires the address from the store server 16. Then,the session server 11 transmits the acquired information and address tothe information processing apparatus 14. By doing in this way, theinformation processing apparatus 14 of the viewer can acquireinformation necessary to configure a viewing screen image only byaccessing the session server 11.

Although the example in which command information is provided from thegame server 12 or the session server 11 is described above, commandinformation may otherwise be provided from the information processingapparatus 10. Especially in such a case that command information changesin response to a game scene, if the information processing apparatus 10transmits command information directly to the information processingapparatus 14, then the viewing user can input an appropriate command forthe game scene. It is to be noted that, where the situation of the gameplay is conveyed on the real time basis to the session server 11, everytime command information is updated, the session server 11 may transmitcommand information to the information processing apparatus 14.

Referring back to FIG. 18, the streaming data reproduction section 322decodes and reproduces streaming data acquired by the streaming dataacquisition section 302, and a live distribution image is displayed inthe live video display region 274. The command image generation section324 generates command images 380 a, 380 b and 380 c based on commandinformation acquired by the command information acquisition section 306,and the command images 380 a, 380 b and 380 c are displayed in thecommand image display region 276. The command information includesinformation for representing command types of the command images 380 a,380 b and 380 c displayed as GUI buttons, and characters “Arrow” aredisplayed on the command image 380 a; “Fire” on the command image 380 b;and “Bomb” on the command image 380 c.

Here, the command image 380 a is a GUI which designates a command forcausing an arrow to fall in the game screen image being live relayed;the command image 380 b is a GUI which designates a command forinjecting fire in the game screen image; and the command image 380 c isa GUI which designates a command for dropping a bomb in the game screenimage. If the viewing user operates the inputting apparatus 6 to selectone of the command images, then the command corresponding to the commandimage is transmitted to the information processing apparatus 10 of thedistribution user through the game server 12. Thus, the game “SAMURAI”in the information processing apparatus 10 executes a process ofreflecting the command on the progress of the game. Consequently, aninteractive game environment is implemented. It is to be noted that awidget for sharing impressions or evaluation of users such as a “like!”button may be prepared.

The purchase mark generation section 326 generates a purchase imagebased on the address of the sale site acquired by the store informationacquisition section 308, and the purchase image is displayed in thepurchase mark display region 280. Here, the purchase image is a Buy mark384, and the address of the sale site is linked to the purchase markdisplay region 280. If the viewing user operates the inputting apparatus6 to select the Buy mark 384, then the information processing apparatus10 connects to the address of the sale site, and a purchase screen imageis displayed on the outputting apparatus 15. It is to be noted that, ifthe store information acquisition section 308 fails to acquire theaddress of the sale site, then the Buy mark 384 is not displayed in thepurchase mark display region 280. Further, where the game is installedalready in the information processing apparatus 14, the Buy mark 384 isnot displayed. The purchase mark generation section 326 confirms theinstallation situation of the game and, when the game is installedalready, the purchase mark generation section 326 does not display theBuy mark 384 in the purchase mark display region 280. It is to be notedthat, in this case, for example, a “Play” button for starting up thegame may be displayed such that, if the “Play” button is selected, thenthe application execution unit 106 starts up the game so that that theviewing user can play the game.

The participation mark generation section 328 generates a participationimage based on session information acquired by the session informationacquisition section 310, and the participation image is displayed in theparticipation mark display region 278. Here, the participation image isa Join mark 382, and the address (URI) of the session is linked to theparticipation mark display region 278. If the viewing user operates theinputting apparatus 6 to select the Join mark 382, then the applicationexecution unit 106 starts up the “SAMURAI” game in a state in which itparticipates in the session. Consequently, the viewing user canparticipate in the game of the distribution user from the viewing screenimage depicted in FIG. 18. It is to be noted that, if the sessioninformation acquisition section 310 fails to acquire the sessioninformation, then the Join mark 382 is not displayed in theparticipation mark display region 278.

The comment image generation section 330 generates a comment image basedon comment data acquired by the comment data acquisition section 312,and the comment image is displayed in the comment display region 286.Preferably, comment data from a plurality of users who access thesharing server 9 are successively displayed in the comment displayregion 286. This makes it possible for the viewing user to browsecomments synchronized with a live video displayed in the live videodisplay region 274, and a game video distribution environment whichprovides a realistic sensation is implemented.

A selection process of a command image 380 is described. In the commandprocessing unit 340, the command input acceptance section 342 accepts aselection input of the command image 380 by the user. The user wouldplace a focus frame at a desired command image 380 in the command imagedisplay region 276 and depress the determination key of the inputtingapparatus 6 to select the command image 380. It is to be noted that theuser may otherwise move the cursor to the command image 380 and depressthe determination key of the inputting apparatus 6 to select the commandimage 380. The command transmission section 344 transmits the commandaccepted by the command input acceptance section 342 to the game server12 together with the distributor specification information. The gameserver 12 recognizes that the transfer destination is the user A fromthe distributor specification information and transmits a command to theinformation processing apparatus 10 of the user A.

It is to be noted that, if the session server 11 manages the addressinformation of the information processing apparatus as described aboveand the information processing apparatus has acquired the addressinformation, then also it is possible for the command transmissionsection 344 to transmit the command to the information processingapparatus 10. It is to be noted that the command transmission section344 may transmit the command not to the game server 12 but to thesession server 11 such that the session server 11 transmits the commandto the information processing apparatus 10. Although, in the informationprocessing system 1, various transmission routes of a command areavailable in this manner, any route may be adopted if a command reachesthe information processing apparatus 10 therealong.

In the information processing apparatus 10, the game accepts a commandand reflects the command on game processing. For example, if the commandimage 380 a is selected on the viewing screen image depicted in FIG. 18,then the “Arrow” command is transmitted to the information processingapparatus 10, and the game reflects the command on game processing. FIG.20 depicts an example of a game screen image in which an arrow comesflying. In this manner, in the information processing system 1, the usernot only can view a live video but also can participate in a game from aviewing screen image. Thus, an interactive game execution environment isgenerated.

A command inputted from a live video viewing screen image in the presentembodiment may be any command which is reflected on game processing, andthe type of the command is not fixed. For example, where choices A to Dto an answer are available in a quiz game, viewers may select thosechoices which are individually believed correct and transmit the choicesto the information processing apparatus 10. At this time, suchproduction that rates of the choices A to D are displayed on the gamescreen image is carried out. Alternatively, the command may be such thatthe game causes the game screen image to display supporting messagesfrom viewers.

A comment inputting process is described. In the comment processing unit360, the comment input acceptance section 362 accepts an input of acomment by the viewing user. Characters accepted by the comment inputacceptance section 362 are suitably displayed in the comment input field284 so that the viewing user can confirm them. If the viewing usercompletes the comment input and then selects the comment mark displayregion 282, then the comment transmission section 364 transmits thecomment data to the sharing server 9. The sharing server 9 accepts thecomment data from the viewing user and distributes the comment data tothe viewing user. It is to be noted that the distributed comment data isacquired by the comment data acquisition section 312 in the informationprocessing apparatus 10 and displayed in the comment display region 286as described hereinabove.

In the present embodiment, the information processing apparatus 14 ofthe viewing user may not have the function of the information processingapparatus 10 but may be a terminal apparatus having the Web browserfunction as described hereinabove. In the case where the informationprocessing apparatus 14 does not have a viewing application, althoughthe information processing apparatus 14 can display a live video, acomment and so forth on the viewing screen image, it cannot communicatewith the game server 12. Therefore, the information processing apparatus14 cannot display the command image 380 and cannot directly transmit acommand to the game server 12.

However, comment data inputted to the comment input field 284 istransmitted to the sharing server 9 by the comment transmission section364 and then transferred from the sharing server 9 to the informationprocessing apparatus 10 of the distribution user. Therefore, in theinformation processing system 1, when the viewing user inputs a commandto the comment input field 284, the comment transmission section 364transmits the inputted command as comment data to the distributor to theinformation processing apparatus 10 through the sharing server 9 suchthat, in the information processing apparatus 10, the game extracts thecommand from the comment data and reflects the command on processing ofthe game. In particular, in the information processing apparatus 10depicted in FIG. 6, the comment image generation section 116 extracts apredetermined character string associated with the command and passesthe predetermined character string to the game so that the game cangenerate a game screen image on which the command is reflected. Thepredetermined character string preferably is a character string which isnot used in an ordinary comment and may be, for example, a commanddelimited by three double quotations (“ ”) on the front side and twodouble quotations (“ ”) on the rear side. For example, if the commentimage generation section 116 detects “““Arrow”” in comment data, then itmay pass a character string between the three double quotations and thetwo double quotations to the game so that the game can interpret thecharacter string as a command.

The present invention has been described based on the embodimentthereof. This embodiment is illustrative, and it can be recognized bythose skilled in the art that various modifications are possible incombination of the constituent elements and the processes of theembodiment and that also such modifications are included in the scope ofthe present invention.

FIG. 21 depicts another example of a viewing screen image displayed onthe outputting apparatus 15. The image generation unit 320 generatesviewing image data including a live video in accordance with a template288 and outputs the viewing image data to the outputting apparatus 15.Compared with the viewing screen image depicted in FIG. 18, in theviewing screen image depicted in FIG. 21, the command image displayregion 276 for displaying the command image 380 is not provided, but acommand mark display region 292 is provided instead. In the viewingscreen image depicted in FIG. 21, since the command image display region276 is eliminated from the display region of the outputting apparatus15, the live video display region 274 can have a greater area than inthe case of the viewing screen image depicted in FIG. 18.

As described hereinabove, the command information acquisition section306 acquires command information to an application based on applicationspecification information. Here, the command information is datanecessary to display information (image) for specifying the command whenthe focus frame is placed on the command mark display region 292.

FIG. 22 depicts an example of a command image to be displayed. If thefocus frame is placed on the command mark display region 292, then thecommand image generation section 324 generates command images 380 a, 380b and 380 c based on the command information acquired by the commandinformation acquisition section 306, and the command images 380 a, 380 band 380 c are displayed in the proximity of the command mark displayregion 292. It is to be noted that, although, in the present example,the command images 380 a, 380 b and 380 c are displayed above thecommand mark display region 292, the command images 380 a, 380 b and 380c may otherwise be displayed at a position at which they do not overlapwith the live video display region 274 (for example, on the right sideof the command mark display region 292).

It is to be noted that, although it is described that, in the presentmodification, command information acquired by the command informationacquisition section 306 is used to display a command image when thefocus frame is placed on the command mark display region 292, thecommand image 380 may otherwise be displayed without using it as acondition that the focus frame is placed on the command mark displayregion 292. In this case, the command image 380 is displayedindependently of the placement of the focus frame, and accordingly, thecommand image 380 is displayed preferably at a position at which it doesnot overlap with the live video display region 274. If the user placesthe focus frame on one of the command images 380 and presses thedetermination key of the inputting apparatus 6 to select the commandimage 380, then the command input acceptance section 342 accepts theselection input of the command image 380 and the command transmissionsection 344 transmits the command to the game server 12 together withdistributor specification information.

It is to be noted that, even if the user watches the command image 380,the user may not possibly be able to recognize a method of use of thecommand image 380 immediately. Therefore, if the focus frame is placedon a command image 380, then the command image generation section 324may display explanation information of the command. This explanationinformation may be acquired by the command information acquisitionsection 306 together with the command information. For example, theexplanation information of “Arrow” is an explanatory text “This Arrowcommand can be used to shoot an arrow from above,” and the explanationinformation of “Bomb” is an explanatory text “This Bomb command can beused to drop a bomb from above.” Where such an explanatory text isdisplayed when the focus frame is placed on a command image 380, theuser can know the substance of the command. Such an explanatory text canbe displayed similarly also when the focus frame is placed on a commandimage 380 in the command image display region 276 depicted in FIG. 18.

It is to be noted that not only an explanatory text of each command butalso an explanatory text of a button itself may be displayed. It hasbeen described that, in FIG. 22, when the focus frame is placed on acommand button in the command mark display region 292, three kinds ofcommand images 380 are displayed. However, together with the commandimages 380, an explanatory text of a command button itself, for example,an explanatory text “It is possible to attack the enemy character withvarious weapons” may be displayed. This explanatory text is placed abovethe command image 380 a. Consequently, the user can know thesignificance of the command group and can know the significance of eachcommand by applying the focus frame to the command.

It is to be noted that, in the description of the embodiment, it isdescribed that, if a viewing user inputs a command to the comment inputfield 284, then the comment transmission section 364 can transmit theinputted command as comment data to a distributor to the informationprocessing apparatus 10 through the sharing server 9. If the userselects a command image 380 in the viewing screen image depicted in FIG.22 or 18, then the comment transmission section 364 may transmit commentdata associated with the command image 380 to the information processingapparatus 10. If a command image 380 is selected in this manner, then apredetermined character string corresponding to the command may betransmitted as comment data to the information processing apparatus 10,and the information processing apparatus 10 may extract the characterstring so that the game can interpret the character string as a command.

It is to be noted that, as described in the embodiment, the informationprocessing apparatus 14 may be a terminal apparatus which does not havethe function of the information processing apparatus 10 but has the Webbrowser function. In this case, if the viewing user inputs a command tothe comment input field 284, then the comment transmission section 364transmits the inputted command as comment data to the distributor to theinformation processing apparatus 10 through the sharing server 9. Inorder to let the viewing user know which command can be inputted, forexample, the information processing apparatus 10 may notify theinformation processing apparatus 14 of a command group which can beinputted so that the Web browser of the information processing apparatus14 can display the command group. In this example, the command groupincludes “Arrow,” “Fire” and “Bomb,” and the information processingapparatus 10 issues a notification of the command group. Thus, theviewing user can grasp which one of the commands should be inputted tothe comment input field 284. It is to be noted that, as described above,the information processing apparatus 10 may issue a notification also ofexplanation information of each command similarly so that theinformation processing apparatus 14 can display the substance of thecommands by explanation texts.

REFERENCE SIGNS LIST

1 . . . Information processing system, 4 . . . Outputting apparatus, 5 .. . Network server, 6 . . . Inputting apparatus, 9 . . . Sharing server,10 . . . Information processing apparatus, 11 . . . Session server, 12 .. . Game server, 13 . . . Management server, 14 . . . Informationprocessing apparatus, 15 . . . Outputting apparatus, 16 . . . Storeserver, 100 . . . Processing part, 102 . . . Communication unit, 104 . .. Acceptance unit, 106 . . . Application execution unit, 110 . . . Imagegeneration unit, 112 . . . Game image generation section, 114 . . .Picked up image generation section, 116 . . . Comment image generationsection, 118 . . . Status image generation section, 120 . . . Menu imagegeneration section, 130 . . . Image changeover unit, 150 . . . Sharingprocessing unit, 152 . . . Input image generation section, 154 . . .Confirmation processing section, 156 . . . Encoding section, 158 . . .Distribution processing section, 160 . . . Template retaining section,162 . . . Information acquisition section, 300 . . . Acquisition unit,302 . . . Streaming data acquisition section, 304 . . . Specificationinformation acquisition section, 306 . . . Command informationacquisition section, 308 . . . Store information acquisition section,310 . . . Session information acquisition section, 312 . . . Commentdata acquisition section, 314 . . . Content information acquisitionsection, 320 . . . Image generation unit, 322 . . . Streaming datareproduction section, 324 . . . Command image generation section, 326 .. . Purchase mark generation section, 328 . . . Participation markgeneration section, 330 . . . Comment image generation section, 332 . .. List image generation section, 340 . . . Command processing unit, 342. . . Command input acceptance section, 344 . . . Command transmissionsection, 360 . . . Comment processing unit, 362 . . . Comment inputacceptance section, 364 . . . Comment transmission section, 370 . . .Template retaining unit

INDUSTRIAL APPLICABILITY

The present invention can be utilized in a technical field in whichstreaming data is distributed or viewed.

1. An information processing apparatus comprising: a first acquisitionunit configured to acquire streaming data including an applicationimage; a second acquisition unit configured to acquire commandinformation to an application based on specification information of theapplication; an image generation unit configured to reproduce thestreaming data to generate an application image and generate a commandimage based on the command information; an acceptance unit configured toaccept a command input to the application in response to selection ofthe command image; and a transmission unit configured to transmit theinputted command.
 2. The information processing apparatus according toclaim 1, further comprising a third acquisition unit configured toacquire specification information of a distributor who provides thestreaming data and specification information of the application, whereinthe transmission unit transmits the inputted command based on thespecification information of the distributor.
 3. The informationprocessing apparatus according to claim 1, wherein the secondacquisition unit acquires the command information from a server.
 4. Theinformation processing apparatus according to any one of claim 1,further comprising a fourth acquisition unit configured to acquireinformation relating to purchase of the application based on thespecification information of the application, wherein the imagegeneration unit generates a purchase image based on the purchaseinformation to generate image data including the purchase image and theapplication image.
 5. The information processing apparatus according toclaim 1, wherein the transmission unit transmits the inputted command ascomment data to a distributor to a server.
 6. An information processingapparatus comprising: an execution unit configured to execute anapplication; an image generation unit configured to generate image datato be displayed on an outputting apparatus; a sharing processing unitconfigured to transmit the image data generated by the image generationunit; and an acquisition unit configured to acquire information relatingto the image data, wherein the image generation unit generates imagedata including an application image and an image of the informationacquired by the acquisition unit, and the sharing processing unitencodes and transmits the image data to be displayed on the outputtingapparatus.
 7. The information processing apparatus according to claim 6,wherein the acquisition unit acquires comment data as the informationrelating to the image data, the image generation unit generates imagedata including the application image and a comment image, and thesharing processing unit encodes and transmits the image data includingthe application image and the comment image.
 8. The informationprocessing apparatus according to claim 6, wherein the image generationunit generates image data including a picked up image supplied from animage pickup apparatus, and the sharing processing unit encodes andtransmits the image data including the application image and the pickedup image.
 9. The information processing apparatus according to claim 6,further comprising an image changeover unit configured to change over animage to be displayed on the outputting apparatus between a menu imageand the application image, wherein the sharing processing unittransmits, when the menu image is displayed on the outputting apparatus,image data which does not include the menu image.
 10. A non-transitory,computer readable recording medium containing a computer program forcausing a computer to carry out actions, comprising: acquiring streamingdata including an application image; acquiring command information to anapplication based on specification information of the application;reproducing the streaming data; generating a command image based on thecommand information; accepting a command input to the application byselection of the command image; and transmitting the inputted command.11. A non-transitory, computer readable recording medium containing acomputer program for causing a computer to carry out actions,comprising: executing an application; generating image data to bedisplayed on an outputting apparatus; transmitting the generated imagedata; and acquiring information relating to the image data; the imagedata including an application image and an image of the acquiredinformation; the image data to be displayed on the outputting apparatusbeing encoded and transmitted.
 12. (canceled)
 13. The informationprocessing apparatus according to claim 6, wherein the sharingprocessing unit generates an input image which indicates choicesregarding whether or not microphone sound is to be included indistribution of the image data or choices regarding whether or not apicked up image of a camera is to be included in distribution of theimage data.
 14. The information processing apparatus according to claim6, wherein the sharing processing unit generates an input image whichindicates choices of a distribution service of the image data.
 15. Aninformation processing system comprising: a sharing server configured toprovide a sharing service; a first information processing apparatusconfigured to transmit an application image to the sharing server; asecond information processing apparatus configured to acquire andreproduce streaming data including the application image from thesharing server; and a management server configured to process anapplication session, wherein the second information processing apparatusacquires command information to an application from the managementserver to display a command image and transmits a command to themanagement server in response to selection of the command image, and themanagement server transmits the command to the first informationprocessing apparatus.